Add SessionFs sqlite support for runtime sqlite routing#1299
Conversation
8c3645d to
efdbbaf
Compare
e9160cf to
28b6443
Compare
Update SDK to support the new sqlite() method on SessionFs, allowing SDK apps to intercept per-session SQLite operations. Key changes: - SessionFsProvider.sqlite() accepts queryType parameter - SessionFsSqliteQueryType exported from all entry points - handleSqlite flag on SessionFsConfig - Regenerated types for all language SDKs (Node, Python, Go, C#, Rust) - E2E test for sqlite round-trip with queryType dispatch Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sqlite is now a required method on SessionFsProvider. The handleSqlite opt-in flag is removed from SessionFsConfig and the setProvider call. Regenerated types for all languages from updated runtime schema. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update generated types, provider interface, and adapter for the flattened sqlite API. Add SessionFsSqliteProvider interface for structured sqlite operations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add ISessionFsSqliteProvider interface for optional SQLite support - Add SessionFsSqliteResult type for provider query results - SessionFsProvider base class delegates to ISessionFsSqliteProvider at runtime - Add Capabilities property to SessionFsConfig - Client validates capabilities.sqlite matches ISessionFsSqliteProvider impl - Regenerate Rpc.cs and SessionEvents.cs from updated schemas - Fix pre-existing SwitchToAsync signature mismatch from codegen - Add Microsoft.Data.Sqlite dependency for tests - Add SessionFsSqliteE2ETests sharing Node SDK snapshots Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ISessionFsSqliteProvider interface pattern replaces the old abstract methods. Remove leftover overrides in test providers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace disk-based file operations with ConcurrentDictionary-backed in-memory store, eliminating providerRoot temp dirs and cleanup code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the in-memory SessionFsProvider + ISessionFsSqliteProvider test implementation and SqliteCall record into a separate file for reuse. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…owid to long? Rows are now positional arrays aligned with the Columns list, avoiding redundant dictionary key construction. The bridge code converts to keyed dictionaries for the wire format. LastInsertRowid changed from double? to long? since SQLite row IDs are always integers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
079a647 to
9c6d4c0
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Existing sessionFs tests used the old flat sqliteQuery/sqliteExists
methods. Updated to use the new sqlite: { query, exists } interface
shape, and added ISessionFsSqliteProvider to ThrowingSessionFsProvider.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The sqliteQuery and sqliteExists adapter methods were throwing errors directly instead of catching them and returning error results like all other SessionFsHandler methods. This caused test failures when providers threw exceptions that should have been caught and mapped to SessionFsError. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit 48a810d.
This comment has been minimized.
This comment has been minimized.
6f7a1d9 to
0a5f123
Compare
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for issue #1299 · ● 1M
- Remove session_id parameter from sqlite_query/sqlite_exists (providers are already session-scoped) - Add clean SessionFsSqliteQueryResult types without error field (errors signaled by raising/returning errors, not via result field) - Update adapters to convert clean result types to generated RPC types - Update all tests to use new signatures and clean result types Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…NET) All SDKs now use (queryType, query, params) parameter order, consistent with the Node.js and .NET reference implementations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…: ignore comments - Rename 'provider' to 'fs_provider' in client.py create/resume to avoid shadowing ProviderConfig typed variable - Import SessionFsProvider in client.py for proper type annotation - Remove 11 unused '# type: ignore[attr-defined]' comments in adapter Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for issue #1299 · ● 1.6M
| public IList<string> Columns { get; set; } = []; | ||
|
|
||
| /// <summary>For SELECT: rows as positional arrays aligned with <see cref="Columns"/>. For others: empty.</summary> | ||
| public IList<object?[]> Rows { get; set; } = []; |
There was a problem hiding this comment.
Cross-SDK consistency suggestion: The .NET provider-facing row type uses positional arrays (object?[]), but every other SDK uses named key-value maps:
| SDK | Rows type in provider-facing result |
|---|---|
| Node.js | { [k: string]: unknown }[] – named map |
| Python | list[dict[str, Any]] – named dict |
| Go | []map[string]any – named map |
| Rust | Vec<HashMap<String, serde_json::Value>> – named map |
| .NET | IList<object?[]> – positional array ← inconsistent |
Using positional arrays is more error-prone for SDK users (column order must match Columns exactly), and it means the adapter has to post-process the data into a keyed form before serialization (lines ~294–304). It would be more consistent and simpler to use named rows directly:
// Instead of positional arrays:
public IList<object?[]> Rows { get; set; } = [];
// Use named rows (consistent with other SDKs):
public IList<IDictionary<string, object?>> Rows { get; set; } = [];If this change is made, the adapter's conversion block that builds keyedRows from positional data (~lines 294–304) could be removed, since the provider would already return named rows directly.
The Columns field can be kept for its own value (e.g., to know column order in SELECT results), just like the other SDKs retain it.
There was a problem hiding this comment.
The row type intentionally varies per language based on what's idiomatic and efficient:
- .NET:
IList<object?[]>— positional arrays are natural (ADO.NET readers give you positional data). The adapter converts to keyed maps for the wire format. - TypeScript:
Record<string, unknown>[]— dicts are the natural JS type. - Go/Python:
map[string]any/dict[str, Any]— natural map types, match the wire format directly. - Rust:
Vec<HashMap<String, Value>>— matches the wire format (generated RPC type), so the adapter passes rows through with zero conversion.
Changing Rust to positional arrays would mean the adapter has to rebuild a HashMap for every row on every query — pointless overhead since the wire format requires keyed maps anyway. Each language's choice minimizes conversion while staying idiomatic.
Go/Python/Rust: Fix E2E stub providers to record actual session ID from stored state instead of hardcoded strings, so filterCalls assertions match the real session. Node.js: Move SQLite database Map to describe scope so it survives handler re-creation. The factory-scoped closure variable caused each handler invocation to create a fresh :memory: database, losing previously inserted data. This caused intermittent failures on Windows where the CLI re-creates the handler between tool calls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change SessionFsSqliteResult.Rows from IList<object?[]> (positional arrays) to IList<IDictionary<string, object>> (keyed dictionaries), matching the generated wire type exactly. This eliminates the positional-to-keyed conversion in the adapter. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for issue #1299 · ● 823.4K
| /// Same shape as <see cref="SessionFsSqliteQueryResult"/> but without the <c>Error</c> field, | ||
| /// since providers signal errors by throwing. | ||
| /// </summary> | ||
| public class SessionFsSqliteResult |
There was a problem hiding this comment.
Cross-SDK naming inconsistency: This class is named SessionFsSqliteResult, but all other SDKs name the equivalent type SessionFsSqliteQueryResult:
- Go:
SessionFsSqliteQueryResult(go/session_fs_provider.go) - Node.js:
SessionFsSqliteQueryResult(nodejs/src/sessionFsProvider.ts) - Python:
SessionFsSqliteQueryResult(python/copilot/session_fs_provider.py) - Rust:
SessionFsSqliteQueryResult(rust/src/session_fs.rs) - .NET:
SessionFsSqliteResult← missing "Query"
Suggestion: Rename to SessionFsSqliteQueryResult to align with the other SDKs. This is a public API type and the name divergence could confuse developers working across languages.
| public long RowsAffected { get; set; } | ||
|
|
||
| /// <summary>Last inserted row ID (for INSERT).</summary> | ||
| public long? LastInsertRowid { get; set; } |
There was a problem hiding this comment.
Cross-SDK type inconsistency: LastInsertRowid is typed as long? here, but all other SDKs use a floating-point type for this field:
- Go:
*float64 - Rust:
Option<f64> - Python:
float | None - Node.js:
number | undefined(viaOmit<GeneratedSqliteQueryResult, "error">) - The generated wire-format RPC type in .NET itself also uses
double?(seeSessionFsSqliteQueryResultinRpc.cs)
Using long? is semantically reasonable since SQLite rowids are always integers, but it diverges from the rest of the SDK surface. It also means there's an implicit long → double widening in the adapter (result?.LastInsertRowid at line 297), which could lose precision for rowids > 2^53 on paper (unlikely in practice).
Suggestion: Use double? to match the wire type and other SDKs. If you prefer integer semantics, consider adding a // SQLite rowids are always integers; cast is safe comment in the adapter.
| // message in the JSON-RPC error response. | ||
| sqliteQuery: async ({ queryType, query, params: bindParams }) => { | ||
| if (!provider.sqlite) { | ||
| throw new Error("SQLite is not supported by this provider"); |
There was a problem hiding this comment.
Cross-SDK behavioral inconsistency: When provider.sqlite is absent (unsupported), this throws an error — which propagates as a JSON-RPC error response to the runtime. The other three SDKs return a structured in-band response instead:
- Go:
return &rpc.SessionFsSqliteQueryResult{ Error: &rpc.SessionFsError{Code: UNKNOWN, Message: "SQLite is not supported..."} }, nil - Python:
return _GeneratedSqliteQueryResult(... error=SessionFSError(..., message="SQLite is not supported...")) - .NET:
return new SessionFsSqliteQueryResult { Error = new SessionFsError { ... } }
The same divergence applies to sqliteExists at line 204: the other SDKs return {exists: false} while this throws.
The comment above explains the rationale for letting errors propagate (FS errno mapping isn't meaningful for SQL), which is a fine approach for provider-thrown errors. But the "not supported" case is different — it's a capability check, not a provider error, and returning a structured response may give the runtime more useful information than an unexpected JSON-RPC fault.
Suggestion: Consider returning an in-band error (matching the Go/Python/.NET pattern) for the "not supported" path, even if provider-thrown query errors continue to propagate:
sqliteQuery: async ({ queryType, query, params: bindParams }) => {
if (!provider.sqlite) {
return { rows: [], columns: [], rowsAffected: 0, error: { code: "UNKNOWN", message: "SQLite is not supported by this provider" } };
}
// provider errors still propagate
const result = await provider.sqlite.query(queryType, query, bindParams);
return result ?? { rows: [], columns: [], rowsAffected: 0 };
},
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for issue #1299 · ● 1.8M
| # mapping (ENOENT) isn't meaningful for SQL errors and the JSON-RPC layer | ||
| # already handles uncaught exceptions. | ||
| if not isinstance(self._p, SessionFsSqliteProvider): | ||
| return _GeneratedSqliteQueryResult( |
There was a problem hiding this comment.
Behavioral inconsistency with Node.js for the "SQLite unsupported" case:
- Python (here): returns a result-level error (
error=SessionFSError(code=UNKNOWN, ...)) - Node.js:
throw new Error("SQLite is not supported by this provider")— i.e. raises a JSON-RPC protocol error
The Node.js comment explicitly says SQLite errors are intentionally left to propagate to the JSON-RPC layer. But Python takes the opposite approach for the "unsupported" case, returning a result-level error. This means the runtime will observe a different error shape depending on which SDK is in use.
Should this return be a raise to match Node.js?
| }; | ||
| } | ||
| catch (Exception ex) when (!IsCriticalException(ex)) | ||
| catch (Exception ex) |
There was a problem hiding this comment.
Behavioral inconsistency with Node.js and Python for SQL execution errors (same issue as go/session_fs_provider.go):
- Node.js: SQL errors propagate to the JSON-RPC layer (no try/catch)
- Python: SQL errors propagate to the JSON-RPC layer (no try/except)
- .NET (here): SQL errors are caught and returned as result-level
SessionFsError
If the intent is for SQL errors to propagate (preserving original error detail in the JSON-RPC response), this catch block should be removed. If result-level error wrapping is preferred, Node.js and Python should add equivalent catch blocks.
Note: the previous implementation had catch (Exception ex) when (!IsCriticalException(ex)) — the new bare catch (Exception ex) will now also swallow critical exceptions like OutOfMemoryException. This may be intentional given the redesign, but worth double-checking.
| """ | ||
|
|
||
| @abc.abstractmethod | ||
| async def sqlite_query( |
There was a problem hiding this comment.
Naming inconsistency with Node.js and .NET: The methods on the separate SessionFsSqliteProvider class are named sqlite_query and sqlite_exists, but:
- Node.js uses
query()andexists()on itsSessionFsSqliteProviderinterface - .NET uses
QueryAsync()andExistsAsync()on itsISessionFsSqliteProviderinterface - Rust uses
sqlite_query()andsqlite_exists()on itsSessionFsSqliteProvidertrait (same as here)
Since all of these methods live on a type already named SessionFsSqliteProvider, the sqlite_ prefix is redundant. Node.js and .NET chose the shorter names. Having Python/Rust use sqlite_query/sqlite_exists while Node.js uses query/exists and .NET uses QueryAsync/ExistsAsync is an inconsistency in the public API surface.
Suggestion: Rename to query() and exists() in Python (and sqlite_query/sqlite_exists → query/exists in Rust) to align with Node.js and .NET.
| }, nil | ||
| } | ||
| result, err := sp.SqliteQuery(request.QueryType, request.Query, request.Params) | ||
| if err != nil { |
There was a problem hiding this comment.
Behavioral inconsistency with Node.js and Python for SQL execution errors:
- Node.js: SQL errors propagate to the JSON-RPC layer (no try/catch, intentional per code comment)
- Python: SQL errors propagate to the JSON-RPC layer (no try/except, same design)
- Go (here): SQL errors are caught and returned as result-level
SessionFsError
The Node.js code comment says: "Letting exceptions propagate preserves the original error message in the JSON-RPC error response." Go's approach wraps the error in result.Error, which means the runtime sees a different wire format for SQL errors depending on the SDK.
If the intent is to let SQL errors propagate (as in Node.js/Python), this if err != nil block should return an error from the adapter function rather than wrapping it in the result struct. Alternatively, if the Go design of wrapping errors is preferred, Node.js and Python should be updated to match.
| query: str, | ||
| params: dict[str, float | str | None] | None = None, | ||
| ) -> SessionFSSqliteQueryResult: | ||
| ) -> SessionFsSqliteQueryResult: |
There was a problem hiding this comment.
sqlite_query return type inconsistency: -> SessionFsSqliteQueryResult (non-optional) vs Node.js query() returning Promise<SessionFsSqliteQueryResult | undefined> and .NET QueryAsync() returning Task<SessionFsSqliteResult?>.
Node.js and .NET allow the provider to return undefined/null for exec-type queries (DDL/multi-statement with no result set), and the adapter substitutes an empty result. Python doesn't have this, meaning Python providers are required to always return a result object even for exec queries.
Consider updating the return type to SessionFsSqliteQueryResult | None and handling it in the adapter (line ~280) to match the Node.js/``.NET behavior.
| var msg = await session.SendAndWaitAsync(new MessageOptions | ||
| { | ||
| Prompt = | ||
| "Use the sql tool to create a table called \"items\" with columns id (TEXT PRIMARY KEY) and name (TEXT). " + | ||
| "Then insert a row with id \"a1\" and name \"Widget\".", | ||
| }); |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Generated by SDK Consistency Review Agent for issue #1299 · ● 2M
| /// Same shape as <see cref="SessionFsSqliteQueryResult"/> but without the <c>Error</c> field, | ||
| /// since providers signal errors by throwing. | ||
| /// </summary> | ||
| public class SessionFsSqliteResult |
There was a problem hiding this comment.
Cross-SDK naming inconsistency: SessionFsSqliteResult vs SessionFsSqliteQueryResult
All other SDK implementations name the provider-facing query result type SessionFsSqliteQueryResult:
- Node.js:
SessionFsSqliteQueryResult(sessionFsProvider.ts) - Python:
SessionFsSqliteQueryResult(session_fs_provider.py) - Go:
SessionFsSqliteQueryResult(session_fs_provider.go) - Rust:
SessionFsSqliteQueryResult(session_fs.rs)
The .NET type here is named SessionFsSqliteResult, dropping the Query segment. Suggest renaming to SessionFsSqliteQueryResult to keep API naming consistent across the SDK family (and match the doc comment on line 11 which already references SessionFsSqliteQueryResult).
// Suggested rename:
public class SessionFsSqliteQueryResult { ... }| SessionFsFileInfo, | ||
| SessionFsProvider, | ||
| SessionFsSqliteProvider, | ||
| SessionFsSqliteQueryResult, |
There was a problem hiding this comment.
Missing SessionFSSqliteQueryType export
SessionFsSqliteProvider is now exported (line 45), and its abstract sqlite_query method signature requires SessionFSSqliteQueryType as the query_type parameter. However, SessionFSSqliteQueryType itself is not exported from the package root.
Node.js re-exports the equivalent type at the package root (SessionFsSqliteQueryType in index.ts). Python users implementing SessionFsSqliteProvider will need to reach into internal modules (copilot.generated.rpc or copilot.session_fs_provider) to get this type, which is inconsistent.
Suggest adding SessionFSSqliteQueryType to both the from .session_fs_provider import (...) block and __all__:
from .session_fs_provider import (
SessionFsFileInfo,
SessionFsProvider,
SessionFsSqliteProvider,
SessionFsSqliteQueryResult,
+ SessionFSSqliteQueryType, # re-exported from generated.rpc via session_fs_provider
create_session_fs_adapter,
)…unused var fixes - Change LastInsertRowid from float to integer type in Python (int|None), Go (*int64), and Rust (Option<i64>). Adapters convert to wire float types. - Make sqlite_query return optional in Python (| None) and Rust (Option<>), matching Node.js and .NET behavior for exec-type queries. - Fix unused msg variable in Python E2E test. - Remove unused import in Rust E2E test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cross-SDK Consistency Review ✅This PR implements SQLite support across all five SDK implementations (Node.js, Python, Go, .NET, Rust). The changes are well-aligned — the same design decisions are applied consistently everywhere. Consistent across all SDKs ✓
Minor note:
|
Summary
SDK-side support for routing per-session SQLite operations through SessionFs. Requires runtime update that hasn't yet shipped.
Key changes
queryType(exec/query/run)SessionFsConfig— opts in to receiving sqlite calls